Classification of lidar measurements using supervised and unsupervised machine learning methods

Abstract. While it is relatively straightforward to automate the processing of lidar signals, more difficult choose periods “good” measurements process. Groups use various ad hoc procedures involving either very simple (e.g. signal-to-noise ratio) or complex Wing et al., 2018) perform a task that easy train humans but time-consuming. Here, we machine learning techniques sort before processing. The presented method generic and can be applied most lidars. We test using from Purple Crow Lidar (PCL) system located in London, Canada. PCL has over 200 000 raw profiles Rayleigh Raman channels available for classification. classify (level-0) as “clear” sky with strong returns, “bad” profiles, which are significantly influenced by clouds aerosol loads. examined different supervised algorithms including random forest, support vector machine, gradient boosting trees, all successfully profiles. were trained about 1500 each channel, selected randomly nights years. success rate identification above 95 %. also used t-distributed stochastic embedding (t-SNE) method, an unsupervised algorithm, cluster our Because t-SNE data-driven no labelling training set needed, attractive algorithm find anomalies been tested on several measurements. data into meaningful categories. To demonstrate technique, have identify stratospheric layers due wildfires.